Feature Reduction with Inconsistency

Feature selection is a classical problem in machine learning, and how to design a method to select the features that can contain all the internal semantic correlation of the original feature set is a challenge. The authors present a general approach to select features via rough set based reduction, which can keep the selected features with the same semantic correlation as the original feature set. A new concept named inconsistency is proposed, which can be used to calculate the positive region easily and quickly with only linear temporal complexity. Some properties of inconsistency are also given, such as the monotonicity of inconsistency and so forth. The authors also propose three inconsistency based attribute reduction generation algorithms with different search policies. Finally, a “mini-saturation” bias is presented to choose the proper reduction for further predictive designing. DOI: 10.4018/jcini.2010040106 78 International Journal of Cognitive Informatics and Natural Intelligence, 4(2), 77-87, April-June 2010 Copyright © 2010, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. There are two disadvantages for the above approaches. The method, which evaluates the feature set one by one, will destroy the internal semantic relation of the original feature set. And the evaluation for the whole subset of features will lead to low efficiency both in temporal complexity and spatial complexity. To overcome these two problems, many researchers introduce the rough set based reduction into feature selection (Hu, Zhao, Xie, & Yu, 2007; Jelonek, Krawiec, & Slowinski, 1995; Lin & Yin, 2004; Zhong, Dong, & Ohsuga, 2001; Swiniarski & Skowron, 2003). The reduction could preserve the semantic correlation of original features (Jensen & Shen, 2004). In this paper, we address the two weaknesses in traditional feature selection and introduce a new feature selection approach with rough set based reduction. We propose a new concept named inconsistency which is easy to calculate and can evaluate whether the attribute set is a reduct quickly. The rest of this paper is organized as follow: Section 2 presents the definitions and concepts related with inconsistency, some properties of inconsistency are also given in this section; Section 3 proposes three inconsistency based reduction algorithms with different search policies; Section 4 presents the “mini-saturation” bias based reduct selection policy to choose the “optimal” one from multiple reducts for further predictive modeling; and finally we conclude this paper in section 5. 1. RElAtEd dEFFInItIons And ConCEpts Some related definitions and concepts are presented as follow: Definition 1 Positive region, P and Q are two sets in the information system U(C, D), P Q C D , ⊆ ∪ , then the positive region of Q in P, denoted asPOS Q P ( ) , can be calculated as: POS Q PX P X U IND Q ( )